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PREFACE 


From  certain  viewpoints  intercorrelation  (that  is,  correlation  between 
independent  variables)  is  not  a major  problem  in  statistical  analysis.  Routine 
instructions  for  solving  multiple-regression  problems  include  formulas  for  net 
regression  coefficients  and  standard  errors  which  autcanaticaJLly  take  account 
of  the  effects  of  inter correlation.  Nevertheless,  research  workers  are  fre- 
quently surprised  when  two  analyses  showing  nearly  the  same  direct  correlations! 
between  the  dependent  and  each  independent  variable  yield  widely  differing  net  j 
regression  and  mxxltiple  correlation  coefficients. 

Many  of  the  three -variable  calculations  discussed  in  this  paper  were 
developed  by  the  senior  author  in  May  19^7  to  explain  such  happenings  in  a 
precise  way.  Late  in  1952  the  Junior  author  rechecked  and  extended  the  three - 
variable  calculations.  He  also  developed  representative  calculations  for  the 
four-variable  case,  setting  up  the  intercorrelation  formulas  in  a matrix  nota- 
tion which  permits  generalization  to  any  nximber  of  variables. 

The  authors  are  indebted  to  Frederick  V.  Waugh  for  helpful  siiggestions 
on  the  foxar -variable  and  general  cases.  The  four -variable  computations  were 
carried  out  by  Jacqueline  Spiro, 
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EFFECTS  OF  INTERCORREIATION  UPON  MULTIPLE 
CORRELATION  AND  REGRESSION  MEASURES 

Karl  A.  Fox,  Chief,  Statistical,  and  Historical  Research  Branch, 
and  James  F.  Cooney,  Jr.,  Mathematical  Statistician 
Agricultural.  Marketing  Service 

Trained  statisticians  have  known  the  effects  of  intercorrelation  in  a 
general  way  for  some  60  years.  They  have  given  particular  attention  to  the 
extreme  case  ("multicollinearity" ) in  which  two  or  more  independent  variables 
are  so  highly  correlated  that  their  separate  effects  cannot  be  distinguished. 
At  the  other  extr«iie,  where  there  is  no  intercorrelation,  the  effects  of  the 
different  independent  variables  are  strictly  additive. 

Many  users  of  the  graphic  method  of  regression  analysis  know  that  inter- 
correlation  between  independent  variables  tends  to  delay  the  convergence  of 
successive  graphic  approximations  toward  the  mathematical  solution. 

Trained  statisticians  are  also  aware  that  increasing  levels  of  intercor- 
relation are  reflected  in  increasing  standard  errors  of  net  regression  coeffi- 
cients— that  is,  hi^  intercorrelation  tends  to  mean  lowered  reliability  for 
the  individual  regression  constants.  But  apparently  it  is  safe  to^say^that 
most  students  of  elementary  statistics  and  most  persons  who  make  regression 
analyses  as  an  adjunct  to  their  applied  work  have  only  a vague  idea  of  the 
effects  of  intercorrelation  and  frequently  get  results  from  multiple -correla- 
tion analyses  that  they  are  unable  to  explain.  More  concrete  information  on 
the  effects  of  intercorrelation  through  its  whole  range  of  variation  and  not 
merely  at  the  points  0 and  1 is  therefore  expected  to  be  useful.  This  infor- 
mation for  the  three -variable  case  is  shown  by  means  of  charts  and  tables  for 
a number  of  pairs  of  values  of  the  simple  (or  gross)  correlation  coefficients 
between  the  dependent  variable  and  each  of  the  two  independent  variables.  The 
foiir -variable  case  is  treated  for  a more  limited  range  of  numerical  values  but 
in  a notation  which  permits  generalization  of  the  results  to  any  number  of 
variables . 

SUMMARY 

With  given  values  for  r^^  and  r^^^  and  a specified  size  of  sample,  all  of 
the  correlation  measures  in  a three -variable  problem  can  be  expressed  simply 
as  fxmctions  of  r23.  In  certain  cases  as  r23  increases  the  coefficient  of 
multiple  determination  declines  continuously.  In  other  cases  it  trends  down 
to  a minimum  value  and  then  increases.  For  given  values  of  ri2  and  r^^^,  r23 
can  take  only  a limited  range  of  values . In  some  cases  •as  the  degree  of 
Intercorrelation  increases  beyond  a ceirtain  level,  the  "weaker"  of  the  two 
partied  regression  coefficients  changes  sign.  Within  a considerable  range  of 
values  of  r23  in  the  region  of  this  sign  change,  the  value  of  the  correspond- 
ing partial  rejp:*ession  coefficient,  for  samples  of  about  20  observations,  does 
not  differ  significantly  frcmi  zero. 


. 2 


The  four-variable  case  is  more  eomplicated  because  six  siciple  correlation 
coefficients  are  involved.  The  same  approach  can  be  used,  however,  by  speci- 
fying five  of  these  and  allowing  the  sixth  one  to  vary  over  its  entire 

range  of  possible  values.  When  each  of  the  three  simple  intercorrelation 
coefficients  is  very  hi^  the  values  of  the  partial  regi'ession  coefficients 
are  very  unstable  and,  in  samples  of  about  20  observations,  are  smaller  than 
their  standard  error  in  aU  but  a small  portion  of  the  range  of  permissible 
values  of  As  T22  ^2k  smller  values,  the  partial  regression 

coefficients  acquire  greater  stability  and  exceed  their  standard  error  over  a 
large  part  of  the  permissible  range  of 


EFFECTS  OF  INTERCORKSIATION  HI  THE  THREE -VARIABLE  CABE 


The  Approach 


The  general  problem  of  three -variable  regression  analysis  is  to  estimate 
the  values  of  a dependent  variable,  based  on  given  values  of  two  independ- 
ent variables,  X2  ^3*  Assume  that  we  have  already  calculated  the  direct 
correlation  coefficients  (rig  and  ri^)  between  Xi  and  X2  on  the  one  hand  and 
Xi  and  X3  on  the  other  for  a nur^er  of  different  problems.  Svppose  that  sev- 
eral. of  the  analyses,  based  on  entirely  different  sets  of  data,  have  yielded 
the  same  values  of  r-jp  and  3^13 • Nevertheless,  in  each  case,  we  vaay  obtain 
different  values  for  the  multiple  and  partial  correlation  coefficients  and  for 
the  net  regression,  or  beta,  coefficients.  The  coefficient  of  multiple  deter- 
mination, Hi^23^  almost  1.0  down  to  0.5,  or  lower. 

We  then  ask,  Why  do  these  differences  occtar?  In  the  three -variable  case, 
these  variations  can  be  wholly  explained  by  variations  in  the  value  of  the 
intercorrelation  coefficient,  r23,  between  the  independent  variables  X2  and  X^. 


Basic  Formulas 


Several  standard  methods  of  calculating  multiple  correlation  and  re^ssion 
coefficients  start  out  from  the  determinant  of  staple  correlation  coefficients, 
which  for  the  three -variable  case  is  as  follows; 


1 3:12  1^13 

ri2  1 r23  = 1 + 2 ri2  3^23  " ^12 

^13  ^23  ^ \ 


All  the  basic  correlation  and  regression  measures  in  the  three -variable  case 
can  be  derived  from  the  values  of  the  three  sinple  correlation  coefficients, 
with  the  following  exceptions  which  are  trivial  in  the  present  context:  First, 
if  we  '^rish  to  talk  about  net  regression  coefficients  in  terms  of  original  units 
(pomds,  dollars,  and  so  on)  rather  than  nonnalized  or  standard  deviation  units 
we  must  multiply  the  beta  coefficients  by  the  ratio  of  the  standard  deviation 
of  the  dependent  variable  to  that  of  the  particular  independent  variable  con- 
cerned. Second,  the  standard  error  of  the  beta  coefficient,  or  of  the  corre- 
sponding net  regression  coefficient,  is  affected  by  the  number  of  observations 
in  the  sai^le. 


. k - 


It  may  "be  seen  from  these  formulas  that  once  we  have  fixed  the  values  of  rip 
and  the  various  measures  can  be  es^ressed  sis^jly  as  functions  of  r23.  In 
the  charts  and  tables  that  follow,  the  value  of  each  correlation  laeasvire  was 
calculated  for  series  of  values  of  the  intercorrelation  coefficient  r23  cover- 
ing all  or  nearly  all  of  the  range  of  possible  values  of  that  coefficient, 

* given  the  stated  values  of  rj_2  smd  ^ 


Discussion  of  Charts  emd  Tables 


In  figure  1 the  values  of  rip  and  r^^  were  chosen  in  such  a way  that  the 


3^13  ^ 

Ls  OJ 


coefficient  of  multiple  determination  is  0*98  ■vrtien  the  intercorrelation  coeffi- 
cient is  zero.  As  r23  increases,  the  value  of  %f23  ^®clines  continuously, 
approaching  a lower  Imit  of  0.if9  as  r23  approaches  1.  At  this  point  the  var- 
iable X3  adds  nothing  to  the  explanation  of  Xi  that  is  not  already  given  by 
the  single  independent  variable  X2«  The  partial  correlation  coefficient  ri2.3 
decreases  continuously  as  r23  increases,  approachii^  zero  as  r23  approaches  1. 
The  beta  coefficient  also  decreases  through  this  entire  range  and  its  standard 
error  increases.  By  the  time  r23  exceeds  0,7,  the  beta  coefficient  (based  on 
an  asstnaed  20  observations)  is  no  longer  significantly  different  from  zero  at 
'the  commonly  used  5 -Percent  probability  level. 


Figure  2 illustrates  the  fact  that  the  values  of  ri2  and  r^o  set  certain 
limits  upon  the  range  of  values  which  r23  may  take.  Obviously,  if  X2  and  X3 
are  both  closely  correlated  with  X^  they  have  some  degree  of  correlation  with 
each  other.  The  exact  nature  of  the  limits  set  t^n  r23  by  the  values  of  r22 
and  ri3  is  shown  in  appendix  note  1.  In  this  particular  case,  r23  cannot  be 
lower  than  0.62, 


Figure 3 shows  a resixlt_that  may  be  s\nprislng  to  many  applied  workers. 

As  Intercorrelation  increases  beyond  a certain  level,  the  "weaker"  of  the  two 
partial  regression  coefficients  changes  sign  from  positive  to  negative.  This 

In  the  charts  and  tables  that  follow,  rj2  and  ri3  are  always  taken  as 
positive,  and  the  corresponding  values  of  r23  and  other  measures  are  predom- 
inantly iKJSitive.  If  the  same  absolute  values  of  T12  taken  with 

negative  signs,  the  corresponding  values  of  Ri,23^  ^23 ^ ^^12  3*  ^^13  2 

the  same  as  before;  absolute  values  of  ^12, 3^  <^13, 2»  ^12,3  ^13.2 

same  as  before  but  with  the  opposite  sign.  If  we  take  T\g  positive  and  ri3 

negative,  r23  will  be  predminantly  negative.  Values  of  Ri^23*  ^022  3*  ^013  2* 

^12,3  ^^.3  ssun®  "the  first  case  (r^^^  ^13  ^"^h  posi- 
tive); and  the  absolute  values  of  02.3,2  ^13.2  ^ same  as  in  the 

first  case  but  with  opposite  sign.  Finally,  lx  we  take  rip  negative  and  ri3 
positive,  the  values  of  r23  will  be  predominantly  negative;  those  of  Rif23' 

^022  3*  ^^13  2*  ^13*2  ^13,2  ■'^11  "the  same  as  in  the  first  case; ‘and 

the  absolute  values  of  ^12.3  8^^  r^^  3 will  be  the  same  as  in  the  first  case 
but  with  opposite  sign.  As  the  absolute  values  of  all  the  measures  axe  un- 
changed by  these  interchanges  of  signs,  the  figures  tabulated  below  each  chart 
can  be  used  for  all  four  cases  with  appropriate  changes  in  signs. 
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Equations  (2)  through  (7)  define  the  various  correlation  and  regression 
measures  in  terms  of  the  three  siiEple  correlation  coefficients; 


1.23 


^12^  ^ ^13^  ~ ^12  ^13  ^23 

1 - r232 

^ ^12  " ^13  ^23 


23 


^12.3  ^12.3 


Si 


■where  and  Sg  are  the  standard  deviations  of  Xj  and  X2  respectively. 

^3.2 


^13  ~ ^12  ^23 


^23 


^13.2  - ^13.2  • 0^  » 
' 5 


■where  is  the  standard  deviation  of  X^. 


^12.3 


ri2  - r^^^  ^2^ 


13.2 


^13  ' ^12  ^23 
N/(l-r23"^  ^l-ri2^^ 


= Si 


’^12.3 " "^13.2  ’ (1-^13) 


\,^F-i!23  \/3^griori3r^^ia--i-i3^-r23^ 


(l-r|3)  4W- 


^‘‘la.s  “ ®Pi2.3  • Si 


, and 


= wS  . 

13.2  "^13.2  ’ S3 


(2) 


(3) 


(3.1) 


(4.1) 


(5) 


(6) 


(T) 


(7.1) 


(7.2) 
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chan^  in  sign  occurs  at  a value  of  r23  soaaewiiat  abovejbhe  value  of  the  lower 
of  the  two  siaiple  correlation  coefficients,  rx2  Within  a consider- 

able Twage  of  values  of  in  the  region  of  this  sign-change,  the  value  of 
the  correj^onding  beta  coefficient  would  not  differ  significantly  from  2@ro« 
Other  features  illustrated  in  figure  3 are  (l)  that  the  "stronger"  of  the  two 
regression  coefficients  increases  for  a time  as  the  intercorrelation  incr^ses, 
and  (2)  timt  the  coefficient  of  multiple  determination  trends  down  to  a mini- 
mum at  some  value  of  r23  neater  than  the  lower  of  the  two  direct  coefficients 
and  then  increases  again. 

The  characteristics  of  figure  3 r^eated  in  the  data  Blmm  ia  tables 
4 and  5 and  in  figure  4.  Each  shows  minimum  values  2/  for  the  coefficient  of 
multiple  detemination,  Rif23#  stronger  partial  correlation  coefficient, 
^12. 3 > stronger  regression  coefficient,  ^ 22.35  each  shows  a sign 

chaise  for  the  weaker  coefficients,  /?13.2  ^13,2*  spread  between 

ri2  and  ri3  increases,  so  also  does  the  range  of  permissible  values  of  r23» 
When  rj^o  falls  to  0.3^  3^23  take  on  small  negative  values  as  well  as  posi- 
tive values.  If  r23  equals  0.1,  then  r23  can  take  values  slightly  lower  than 

-0.3. 

The  s^^Emary  tables  contain  values  of  the  "t-ratios",  that  is,  ratios  of 
the  rej^ctive  net  re^^ssion  coefficients  to  their  standard  errors.  As  in 
each  of  the  last  four  cases  ^22.3  ^ minlsRaa  and  3 ^ ssaxlmusa,  the 

correE^onding  t-ratio  has  a mlnimjjaa  beyond  -idaich  it  rises  again  with  further 
Increases  in  r23e 


£/  Tlv>^t  jg,  HilnliEta  In  the  math)Mffi.tieal  sense  of  points  at  which  the  slc^ 
with  respect  to  r23  is  zero  and  becomes  positive  as  r23  increases  and  negative 
as  r23  decreases.  Further  informtion  on  these  sdniiffim  values  is  given  ia 
appendix  note  2. 
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Figure  1 


Table 

1.-  Data 

for  case  in  which  r^^  = 

0.7,  rj^2  “ 

0.7>  and  N ■ 

20 

: t -ratio  for  - 

^23 

p2  • 

^1.23  : 

^12.3 

: ^13.2 

: *‘l2.3 

: ^13.2 

: 

: ^ 

; ®12.3 

; ^13.2 

-0.020 

1.0000 

0.7143 

0.7143 

1.0000 

1.0000 

0 

... 

0 

.9800 

.7000 

.7000 

.9803 

.9803 

0.0346 

20.2312 

20.2312 

.100 

.8909 

.6364 

.6364 

.8866 

.8866 

.0806 

7.8958 

7.8958 

.200 

.8167 

.5833 

.5833 

.8003 

.8003 

.1058 

5.5132 

5.5132 

.300 

.7538 

.5385 

.5385 

.7193 

.7193 

.1261 

4.2704 

4.2704 

.iKX) 

.7000 

.5000 

.5000 

.6417 

.6417 

.1449 

3.4507 

3.4507 

.500 

.6533 

.4667 

.4667 

.5659 

.5659 

.1649 

2.8302 

2.8302 

.600 

.6125 

.4375 

.4375 

.4901 

.4901 

.1887 

2.3185 

2.3185 

.700 

.5765 

.4118 

.4118 

.4118 

.4118 

.2209 

1.8642 

1.8642 

.800 

.5W 

.3889 

.3889 

.3267 

.3267 

.2728 

1.4256 

1.4256 

.900 

.5158 

.3684 

.3684 

.2249 

.2249 

.3872 

.9514 

.9514 

.950 

.5026 

.3590 

.3590 

.1570 

.1570 

.5478 

.6553 

.6553 

.9^ 

.49l^9 

.3535 

.3535 

.0985 

.0985 

.8655 

.4084 

.4084 

.990 

.4925 

.3518 

.3518 

.0695 

.0695 

1.2278 

.2865 

.2865 

.999 

.4902 

.3502 

.3502 

.0219 

.0219 

3.1623 

.1107 

.1107 

1.000  iJ 

... 

--- 

... 

--- 

--- 

0 

0 

^ Identical  for  each  P. 

2/  Lowest  possible  value  of  '^2^ 
3/  Highest  possible  value  of  rgg* 
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Table  2.-  Data  for  ceise  In  which  r-j^  = 0.9,  r-j^j  = 0-9>  N = 20 


^23 

r2 

^1.23 

! ^12.3 

: P13.2 

i ^12.3 

^13.2 

i/ 

t-ratlo 

for  - 

^12.3  = 

^13.2 

0.620  2/ 

1.0000 

0.5556 

0.5556 

1.0000 

1.0000 

0 

... 

... 

.700 

.9529 

.5294 

.5294 

.8710 

.8710 

0.0735 

7.2073 

7.2073 

.800 

.9000 

.5000 

.5000 

.6883 

.6883 

.1277 

3.9154 

3.9154 

.900 

.8526 

.4737 

.4737 

-4737 

.4737 

.2135 

2.2183 

2.2183 

.950 

.8308 

.4615 

.4615 

.3306 

.3306 

• 3195 

1.4444 

1.4444 

.980 

.8182 

.4545 

.4545 

.2076 

.2076 

.5193 

.8752 

.8752 

.990 

.8l4l 

.4523 

.4523 

.1463 

.1463 

.7431 

.6087 

.6087 

.999 

.8000 

.4500 

.4500 

.0462 

.0462 

2.0000 

.2250 

.2250 

1.000  ^ 

... 

... 

... 

... 

... 

... 

... 

... 

l/  Identical  for  each  p. 

2/  Lowest  possible  value  of 
Highest  possible  value  of 
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Figure  3 


Table  3-*  Data  for  case  in  which  = 0,9,  r^^j  = 0.7  and  N = 20 


^23 

p2 

^.23 

Pl2.3 

^13.2 

^12.3 

^13.2 

Sp 

1/ 

t-ratio 

^12.3  ! 

for  - 

^13. 2 

0.3187  2/ 

1.0000 

0.7535 

0,1*599 

1.0000 

1.0000 

0 

... 

... 

.hooo 

,91*88 

.7381 

.401*8 

.9473 

.8511 

0,0599 

12.3222 

6.7579 

.1*500 

.9191 

.7335 

.3695 

.9172 

.7578 

.0772 

9.5013 

4.7863 

.5000 

.8933 

.7333 

.3333 

.8892 

.6623 

.0917 

7 .9967 

3.6347 

.5500 

.&703 

.7381* 

.2939 

.8635 

.5632 

.1046 

7.0593 

2.8098 

.6000 

.8500 

.7500 

,2500 

,8402 

.4588 

.1175 

6.3830 

2.1277 

.6500 

.8329 

.7706 

.1991 

.8200 

.3472 

.1305 

5.9050 

1.5257 

.7000 

.8196 

.8039 

.1373 

.8039 

.2249 

.1442 

5.5749 

.9521 

.7500 

.8111* 

.8571 

.0571 

.7938 

.0867 

.1592 

5.3838 

.3587 

.8000 

.8111 

.91*41* 

-.0556 

.7935 

-.0765 

.1758 

5.3720 

-.3163 

.8500 

.8252 

1.0991 

-.231*2 

.8107 

-.2831 

.1925 

5.7096 

-1.2166 

.9000 

.8737 

1.1*211 

-.5789 

.8673 

-.5789 

.1977 

7.1882 

-2.9282 

.91*13  ij 

1.0000 

2.1149 

-1.2912 

1.0000 

-1,0000 

0 

]J  Identical  for  each  p , 

2/  Lowest  possible  value  of 
2/  Highest  possible  value  of  . 
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Table  k„~  Data  for  case  In  which  ==  0»9j>  2*13  0<.5  aad  N = 20 


r 

23 

j %.23 

• 

• 

: 

% 

0 

j 

• 

* 

^12. 3 

^13-2 

3^12.3 

^13  0 2 

H 

1/ 

: t -ratio  for  - 

: ^12. 3 

• 

• 

; P13.2 

♦ 

• 

0 

• 

to 

1.0000 

0.8684 

0.4371 

1.0000 

1.0000 

0 

.1000 

.9798 

.8586 

,4l4l 

.9864 

.9454 

0.0346 

24,8150 

11.9682 

.2000 

»9167 

. .8333 

.3333 

.9428 

.7492 

.0714 

11.6709 

4.6681 

.3000 

,8681 

.8242 

.2527 

.9079 

.5532 

.0922 

8.9393 

2.7408 

,4000 

• 8333 

.8333 

.1667 

.8819 

.3504 

.1082 

7.7015 

1.5407 

.5000 

•8133 

.8667 

.0667 

,8667 

.1325 

.1208 

7.1747 

.5522 

.6000 

.8125 

^9375 

-.0625 

.8661 

-.1147 

.1311 

7.1510 

-.4767 

.7000 

.8431 

1.0784 

-.2549 

.8892 

».4176 

.1345 

8.0178 

-1.8952 

.8000 

0 

1.3889 

»,6lll 

.9623 

-.6413 

.0954 

14.5587 

-6.4057 

.8275  ^ 

1.0000 

1.5428 

-.7766 

1.0000 

-1.0000 

0 

"y  Identical  for 

each  3» 

y Lowest  possible 

value  of 

^23* 

y Highest  possible  value  of  2*23, 

Table  5.- 

Data  for 

case  in  which  ri2  = 

0.9#  2^13  =0.3  and  N = 20 

♦ • 

• • 

: t 

r23  . 1.23  j 

s : 

312.3 

313.2 

0 

^12.3  .* 

^13.2 

Sp 

1/ 

t -ratio 

for  - 

« 

^12.3  : 

: 

^13.2 

• 

-0.1458  2/:  1.0000 

0.9642 

0.4406 

1.0000 

1.0000 

0 

-.1000  : .9636 

.9394 

• 3939 

•9798 

.8992 

0.0465 

20.2022 

8.4710 

0 % .9000 

.9000 

.3000 

.9435 

.6882 

.0767 

11.7340 

3.9113 

.1000  . .8545 

.8788 

.2121 

.9166 

.4842 

.0930 

9.4495 

2.2806 

.2000  j .8250 

.8750 

.1250 

.8987 

.2810 

.1035 

8.4541 

1.2077 

.3000  . .8110 

.8901 

.0330 

,8^1 

.0722 

.1105 

8.0552 

.29^ 

.4000  : .8143 

,9286 

-.0714 

.8921 

-.1502 

.1141 

8.1385 

-.6258 

.5000  s .8400 

1.0000 

-.2000 

.9079 

-.3974 

.1120 

8.9286 

-1.7857 

.6000  : .9000 

1.1250 

-.3750 

.9439 

-.6883 

.0959 

11.7310 

“3-9103 

.6858  1.0000 

1.3107 

-.5988 

1.0000 

-1.0000 

0 

^ Identical  for  each  3. 


^ Lowest  possible  value  of  ^23^ 
^ Highest  possible  value  of  ^23. 
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3-Variabie  Case:  ri2=.9;  ri3  = .l;  N =20 


1 

% 

a 

^\2' 

i 

2.3 

% 

% 

< 

N.-J 

^ f 

13.2 

\ 

a 

1.23 

^ — 1 

s, 

1 

% 

< 

1 

iS 

^ 13.2 

\ > 

-.2  0 .2  .4  .6  .8  1.0 

rjs 
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Figure  h 

Table  6.-  Data  for  case  In  which  “0.9,  r^^  “ 

1,  and  H “ 

20 

^23 

r2 

^1.23 

: ^12. 3 

^13.2 

: ^12.3 

^13.2 

; ^ 

! ^ 

1 t-ratlo  for  - 

s ^12.3 

^13.2 

-O.B^fST  2/ 

1.0000 

1.0595 

0.4641 

1.0000 

1.0000 

0 

— 

... 

-.3000 

.9601* 

1.0220 

.4066 

.9798 

.8899 

0.0510 

20. 0392 

7.9T25 

-.2000 

.8917 

.9583 

.2917 

.9437 

.6556 

.0812 

U.8OI7 

3.5924 

-.1000 

.8465 

.9192 

.1919 

.9192 

.4381 

.1010 

9.1010 

1.9000 

0 

.8200 

.9000 

.1000 

.9045 

.2294 

.1030 

8.7379 

.9709 

.1000 

.8101 

.8990 

.0101 

.8990 

.0231 

.1122 

8.0125 

.0900 

.2000 

.8167 

.9166 

j -.0833 

.9027 

-.1873 

.1058 

8.6635 

-.7873 

.3000 

.8418 

.9560 

-.1868 

.9166 

-.4089 

.1010 

9.4653 

-1.8495 

.1*000 

.8930 

1.0000 

-.3023 

.9319 

-.6432 

.0854 

11.7096 

-3.5398 

• 5000 

.9733 

1.1333 

-.4667 

.9864 

-.9272 

.0458 

24.7445 

10.1900 

.5237  3/ 

1.0000 

1.1680 

-.5116 

1.0000 

-1.0000 

0 

— 

... 

Xj  Identical  for  each  p . 

2/  Lowest  possible  value  of  r23. 
3/  Highest  possible  value  of  rgj* 
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EFFECTS  OF  INTEFCOKREIATTON  IN  THE  FOOT -YARI ABLE  CASE 
Differences  from  the  Three -Variable  Case 


The  problem  of  intercorrelation  in  the  four-variable  case  is  much  more 
complicated  because  there  are  now  three  intercorrelation  coefficients  instead 
of  one.  We  have  three  independent  variables,  X^,  X^,  and  Xj^,  and  we  may  have 

intercorrelation  between  X2  and  X^,  X^  and  Xj^,  and  X^  and  Xj^* 

Following  the  approach  used  in  the  three-variable  case,  let  us  suppose 
that  we  have  a large  number  of  four-variable  regression  analyses  on  different 
sets  of  data.  We  select  a number  of  these  analyses  in  which  the  values  of 
^12’  ^13’  ^lU  direct  or  simple  correlation  coefficients  between  the 

dependent  and  each  independent  variable)  are  about  the  same.  Nevertheless, 
we  find  that  the  partial  correlation  and  net  regression  coefficients  are 
different  in  each  case.  These  differences  are  due  to  the  varying  degrees  of 
Intercorrelation,  represented  by  combinations  of  values  of  ^3^* 

A systematic  exploration  of  the  effects  of  intercorrelation  in  the 
four-variable  case  would  Involve  a great  deal  of  labor.  One  possibility 
wc»uld  be  to  fix  the  values  of  rg^  '^2k  trace  the  effects  of  variations 

in  r^.  upon  the  different  regression  measures.  Except  for  a change  in 
notation,  such  a demonstration  would  apply  equally  well  to  changes  in  either 
of  the  other  intercorrelation  coefficients,  T2-^  or  rg^j^.  Before  doing  this, 
however,  we  shall  illustrate  the  complications  of  the  four-variable  case  in 
terms  of  the  basic  formulas  for  correlation  and  regression  coefficients. 

Basic  Formulas 


It  will  be  convenient  at  this  point  to  introduce  a determinant  notation, 
which  avoids  excessive  rewriting  of  the  simple  correlation  coefficients. 

This  notation  can  be  extended  to  five  or  more  variables,  and  also  to  the 
three -variable  case  previously  considered. 


In  the  three -variable  case,  the  three-rowed  determinant  of  correlation 
coefficients 


12 


ri2 


^13  ^23 


^13 

^23 

1 


(1) 


can  be  made  to  yield  nine  different  two-rowed  determinants  by  deleting  one 
column  and  one  row  of  /§\.  Suppose  we  call  the  two-row  determinant  obtained 
by  deleting  the  first  column  and  the  first  row^^^^  (=  1 - ^23^ obtained 
by  deleting  the  first  column  and  the  second  row,  (=  r^g  - r,-  rg^)!  and 

so  on.  The  complete  set  of  two-row  determinants,  which  we  call  the  @ . is 

as  follows : 
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^11  = ""^23^ 

(1.1) 

A 22  ^ ^ “ ^13^ 

(1.2) 

A 33  = ^ “ ^12^ 

(1.3) 

A 12 " ^12 " ^13  ^23 " A21 

(l.M 

A 13  = - ("13  ■ ^12  "23^=A3i 

(1.5) 

A23  " ^^23  “ ^12  ^13^  " A32 

(1.6) 

formulas  for  correlation  and  regression  measures 

given  in  the 

the  *8,  as  follows: 


^1.23  ■ ^ 


^12.^  ^^12 
' ^ 

^13.2  " “ 


12.3 


A 


12 


13.2 


= 

12.3  *^13.2  11  \/  h-3 


= s 


(2) 


(3) 


(M 


(5) 


(6) 


(7) 


Once  we  have  fixed  the  values  of  r,^  and  r,  , /3\  and  all  hut  two  of 

12  T ^ 


the  *s  are  functions  only  of  r ; these  two,  and^  , are  constants 

• ■ ■ T 1 ^ S 


IJ 


23 


22 


‘33 


Each  of  the  six  formulas  In  this  paragraph  Involve s/^^,  which  changes  with  ^23^ 
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and  another  detemlnant  ^ich  also  changes  with  ^23*  aieans  that  we 

cannot  vary  r^^  arhltrarlly  without  consistently  varying  and  the  ^3 . 

In  the  four -variable  case,  the  determinant  of  correlation  coefficients  is 


A= 


1 

^12 

"13 

"*lh 

^12 

1 

"^23 

^2k 

""13 

"*23 

1 

''3k 

^lU 

^21^ 

"■34 

1 

The  determinant  of  the  three  inter correlation  coefficients  is 


1 r r 

23 


^23  ^ 


^24  3k  1 


3^ 


1 + 2 r, 


23  ^2k  ^3h 


2 2 . 
’ ^23  " "*21^ 


(8) 


(9) 


Each  of  the  I5  other  posslble^^^^ 's  is  now  also  a three-rowed  detennlnant . 
The  formulas  in  the  preceding  paragraph  still  apply,  with  an  appropriate 
change  in  notation:  For  example. 


12. 3t^ 


*‘12 

"23 

CVJ 

""is 

1 

^12  = 

^Ik 

"Zk 

1 

1 

^23 

^24 

1 

r . 

23 

3k 

^2k 

1 

(10) 


All  of  the  *s  for  which  1 V J involve  all  three  intercorrelation 

coefficients.  and/^V^^j^  each  contain  only  one  of  these  coefficients. 

But  this  last  point  is  not  very  helpful,  as  each  fomula  (2)  through  (7)  in* 

eludes  either /^itself  or  a^.  . for  which  i ^ J. 

^ J 
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We  noted  In  the  three-variable  case  that  the  values  assumed  for  r^  and 
1*2^2  impose  certain  limits  upon  the  values  vhich  might  be  assumed  by 
Similarly,  the  values  assumed  for  r^^g  and  r^^j^  restrictions  on  r^j^,  and 

those  assumed  for  r^o  and  r,.  impose  restrictions  on  r,j^.  For  example,  If 
^12  “ ^13  ^14  “ ^ each  of  the  three  intercorrelation  coefficients  may  range 

from  - 0.02  to  1.0,  However,  the  values  of  rgo  and  r^j^  also  set  limits  to  the  , 
permissible  values  of  r^^.  a consistent  set  of  limits  for  the  six  simple  r*s 
can  be  derived  from  the  fact  that/^^g./^oo  . and^il^^ii  must  all  lie 

between  0 and  1 

Discussion  of  Charts  end  Tables 


Figures  5 through  10  and  tables  7 through  13  provide  some  insights  into 
the  effects  of  intercorrelation  in  the  four-variable  case.  The  first  five 
cases  assume  that  all  three  of  the  direct  correlation  coefficients  r^g,  and 

are  equal  to  0.7,  Two  of  the  Intercorrelation  coefficients,  r22  and  rgj^, 
are  then  set  equal  to  0.9^  0.7,  0.5,  0,3  and  0.1,  respectively.  In  each  case, 
the  third  intercorrelation  coefficient,  is  allowed  to  vary  over  its  entire 

range  of  possible  values  given  the  values^of  the  other  five  coefficients  and 
the  basic  requirements  234^  ^ 1^3u|^  ^ 

The  fact  that  we  have  set  all  three  of  the  direct  coefficients  equal  to 
one  another,  and  two  of  the  Intercorrelation  coefficients  equal  to  each  other, 
produces  several  symmetries  in  the  results.  One  is  that|3jL2.2U  ®“^^1U.23 
equal  in  each  case.  Another  is  that  at  the  point  where  r^i^  is  equal  to  r23  and 
r2i^,  all  three  beta  coefficients  are  equal. 

Figure  5 reflects  a very  high  degree  of  Intercorrelation,  One  symptom  of 
this  is  the  fact  that^^jj^,  the  determinant  of  Intercorrelation  coefficients, 
takes  on  very  small  values — O.036  or  less — over  its  entire  range.  In  figure 
10,  in  contrast,  the  value  of^^-^^^  reaches  a peak  of  1.0  when  all  three  inter- 
correlation  coefficients  are  zero,  and  exceeds  0.5  over  a considerable  range  of 
values  of  r^j^.  In  fact,  figure  5 approaches  the  extreme  of  goltleol  1 1 wearity  to 
which  Frisch  gave  so  much  attention  in  the  early  thirties.  The  values  of  the 
beta  coefficients  are  very  unstable  and  are  smaller  than  their  standard  errors 
in  all  but  a small  portion  of  the  range  of  permissible  values  of  r-.  , And  the 
range  of  permissible  values  of  r^j^  is  limited.  ^ 

Figure  6,  in  which  said  rgj^  equal  0.7,  shows  a greater  stability  of  the 

beta  coefficients  with  respect  to  given  changes  in  r-u  than  does  figure  The 
standard  error  of  the  beta  coefficients  is  also  more  "^stable  than  in  the  pre- 
ceding chart.  The  beta  coefficients  exceed  their  standard  errors  over  a 
considerably  wider  range  of  values,  although  they  do  not  reach  twice  the  level 
of  their  standard  errors  anywhere  in  the  permissible  range.  In  both  of  these 
figures  the  behavior  of^^^  corresponds  to  that  of  the  weaker  coefficient 
in  some  of  the  three -vsa*l^le  charts.  When  the  level  of  r^j^  drops  significantly 
below  the  levels  of  T2-^  and  3U  sign  from  positive  to  negative. 

Visually,  it  appears  that  ^13,24  ^lk,23)  » reflection  of  ^22  about  the 

particular  level  at  which  all  three  intercorrelation  coefficiently  hence  all 
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three  teta  coefficients,  are  equal.  The  value  of  the  "betas  at  this  point  of 
equality  Increases  from  one  case  to  the  next  as  the  paired  intercorrelation 
coefficients  (rg^  and  rgi^)  decrease. 

Figure  7 shows  still  greater  stability  in  the  values  of  the  beta 
coefficients  and  their  standard  errors.  The  beta  coefficients  exceed  their 
standard  errors  over  a large  part  of  the  penaissible  range  and  for  certain 
limited  veJ-ues  of  r^i^?  near  zero,  they  exceed  two  standard  errors. 

The  data  given  in  table  10  shw  increasing  stability  of  the  beta 
coefficients  and  their  standard  errors  within  the  range  of  permissible  values. 
However,  that  range  itself  is  somewhat  reduced.  Apparently,  as  rg^  ®ad  r^i^  are 

lowered,  r-.  must  remain  significantly  above  zero  if  the  other  constraints  on 
the  varlous^correlation  measures  are  to  be  met.  If  all  three  intercorrelation 
coefficients  were  zero  the  coefficient  of  multiple  determination,  H i , should 
be  equal  to  the  sum  of  squares  of  the  three  direct  correlation  coeff£ci@ntS”“in 
this  case,  3 ^ (0.7)^  = I.I7.  As  this  is  an  impossible  value  of  bb® 

value  of  r^i^  which  leads  to  it  is  not  permissible.  While  the  ratios  of  the  beta 
coefficients  to  their  standeird  errors  etre  greater  theai  1 over  most  of  the  range 
of  permissible  values,  at  no  point  does  any  one  of  these  ratios  exceed  2.0, 

Figure  8 shows  a still  greater  contraction  of  the  rang®  of  permissible 
values  of  The  values  of  the  beta  coefficients  are  quite  stable  within 

this  limited  range,  but  th©  standard  errors  of  these  coefficients  is  changing 
rapidly  within  it.  The  t-ratlo,  /^/S^  , for  ^^2  exceeds  2.0  toward  th©  lower 

end  of  the  permissible  range  of  t-ratios  for  th©  other  beta  coefficients 

do  not  exceed  1.3  at  any  value  of  r^j^. 

It  is  evident  from  the  above  results  that  to  have  each  of  the  3 direct 
correlation  coefficients  equal  to  0.7  already  constitutes  a high  degree  of 
intercorrelation  if  one  hopes  to  achieve  significant  regression  coefficients 
in  a four-variable  equation  Involving  only  20  or  so  observations. 

In  figure  9>  the  direct  correlation  coefficients  are  reduced  to  0.5  and 
two  of  the  Intercorrelation  coefficients  are  also  set  equal  to  0.5.  This  chart 
may  be  compared  with  that  of  figure  6,  in  which  all  5 of  these  coefficients  were 
set  equal  to  0.7,  The  beta  coefficients  and  their  standard  errors  in  figure  9 
are  considerably  more  stable  and  cover  a wider  range  of  permissible  values  than 
in  figure  6.  The  coefficients  exceed  their  standard  errors  over  most  of  th© 
permissible  range,  and  the  t-ratios  for  23  over  a 

sizable  range,  reaching  a maximum  of  2.83  when  r^j^  reaches  its  lowest  value. 

In  figure  10  the  three  direct  correlation  coefficients  are  again  set  equal 

to  0.5  and  two  of  the  Intercorrelation  coefficients  are  set  at  zero.  The  rang© 

of  permissible  values  of  r^^j^  is  about  the  seuao  as  in  figure  9 but  the  degree  of 

stability  of  the  beta  coefficients  and  their  standard  errors  is  considerably 
greater.  The  coefficient  ^ ^2  34  i®  independent  of  2‘24*  The  t-ratios  are 
greater  than  1 over  almost  the  full  range  of  permissible  values  and  exceed  2.0 
over  considerable  portions  of  this  range. 
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Figure  5 


Table  7.-  Data  for  case  In  which  = ^13  = 
^23  “ ^2h  = N = 20 


^34 

P12.34 

^13.24 

^14.23 

^12.34  .’ 

t -ratio  for 

^13.24 

^14.23 

0.6392  2/ 

-5.8586 

3.6436 

3.6436 

3.0000 

-1.9529 

1.2145 

1.2145 

.6500 

-3.5000 

2.3333 

2.3333 

2.2105 

-1.5834 

1.0556 

1.0556 

.6900 

-1.1000 

1.0000 

1.0000 

1.3474 

-.8164 

.7422 

.7422 

.69^1 

-1.0000 

.9444 

.9444 

1.3102 

. -.7633 

.7208 

.7208 

.7000 

-.8750 

.8750 

.8750 

1.2627 

-.6930 

■ 6930 

.6930 

.7500 

-.2692 

.5385 

.5385 

1.0429 

-.2582 

.5163 

.5163 

.7720 

-.1290 

.4605 

.4605 

1.0000 

-.1290 

.4605 

.4605 

.8000 

0 

.3889 

.3889 

.9721 

0 

.4000 

.4000 

.8500 

.1522 

.3044 

.3044 

.9825 

.1549 

.3098 

.3098 

.8645 

.1847 

.2862 

.2862 

1.0000 

.1847 

.2862 

.2862 

.9000 

.2500 

.2500 

.2500 

1.0827 

.2309 

.2309 

.2309 

.9500 

.3182 

.2121 

.2121 

1.4044 

.2266 

.1510 

.1510 

1.0000 

— - 

... 

... 

1/  Identical  for  each  P. 


2/  Lowest  possible  value  of  r^l^* 
^ Highest  possible  value  of  r2l(.. 
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4-Variable  Case:  ri2  = ri3  = ri4  = .7;  r23  = r24  = .7;  N = 20 
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Figure  6 


Table  8.-  Data  for  case  in  which  = ^^2^4  = 0.7) 

^23  “ ^24  “ 0.7,  and  N = 20 


^12.311 

P 13.24 

P 14.23 

; ^ 

: ^ 

t-ratio  for 

- 

= Pi2.34 

• P 13.24 

; P 14.23 

0.1529  2/ 

-1.0000 

1.2143 

1.2143 

0.6532 

-1.5310 

1.8590 

1.8590 

.1900 

- .7000 

1.0000 

1.0000 

.5782 

-1.2106 

1.729*^ 

1.7294 

.2000 

- .6364 

.95*^6 

.95*f6 

.5625 

-1.1314 

1.6971 

1.6971 

.3000 

- .2188 

.6562 

.6562 

.4622 

- .4733 

1.4199 

1.4199 

.Uooo 

0 

.5000 

.5000 

.4167 

0 

1.2000 

1.2000 

.5000 

.1346 

.4038 

.4038 

.3982 

.3381 

1.0142 

1.0142 

.5765 

.2071 

.3521 

.3521 

.3973 

.5214 

.8862 

.8862* 

.6000 

.2258 

.3387 

.3387 

.3992 

.5657 

.8485 

.8485 

.7000 

.2917 

.2917 

.2917 

.4210 

.6928 

.6928 

.6928 

.8000 

.3415 

.2561 

.2561 

.4772 

.7155 

.5367 

.5367 

.9000 

.3804 

.2283 

.2283 

.6309 

.6030 

.3618 

.3618 

.9592 

.3998 

.2145 

.2145 

.9520 

.4199 

.2253 

.2253 

.9632 

.4010 

.2136 

.2136 

1.0000 

.4010 

.2136 

.2136 

.9800 

.4060 

.2100 

.2100 

1.3440 

.3021 

.1562 

.1562 

1.0000  2/ 

... 

1/  Identical  for  each  g . 

2/  Lowest  possible  value  of  r^i^. 
jJ  Highest  possible  value  of  r^j^. 
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Figure  7 


Table  9.-  Data  for  case  In  which  «■  <•  « 0.7, 

^23  “ ^2k  “ H = 20 


^34 

t 

^12.34  ; 

^13.24 

^14.23 

Sp 

1/ 

^12.34 

t-ratlo  for  - 

: ^13.24  • 

^14.23 

-0.0196  2/ 

-0.0286 

0.7286 

0.7286 

0.3572 

-0.0600 

2.0396 

2.0396 

0 

0 

.7000 

.7000 

.3500 

0 

2.0000 

2.0000 

.1000 

.1167 

.5833 

.5833 

.3224 

.3618 

1.8091 

1.8091 

.2000 

.2000 

.5000 

.5000 

.3062 

.6532 

1.6330 

1.6330 

.2500 

.2333 

.4667 

.4667 

.3012 

.7746 

1.5492 

1.5492 

.3000  : .2625 

.4375 

.4375 

,2981 

.8806 

1.4676 

1.4676 

.4000  : .3111 

.3889 

.3889 

.2970 

1.0474 

1.3093 

1.3093 

.5000  : .3500 

.3500 

.3500 

.3031 

1.1547 

1.1547 

1.1547 

.6000 

.3818 

.3182 

.3182 

.3182 

1.2000 

1.0000 

1.0000 

.7000 

.4083 

.2917 

.2917 

.3472 

1.1762 

.8402 

.8402 

.8000 

.4308 

.2692 

.2692 

.4038 

1.0667 

.6667 

.6667 

.9000 

.4500 

.2500 

.2500 

.5449 

.8259 

.4588 

.4588 

.9500 

.4586 

.2414 

.2414 

.7538 

.6084 

.3202 

.3202 

.9800 

.4635 

.2365 

.2365 

1.1763 

.3940 

.2010 

.2010 

1.0000  3/ 

... 

••• 

••• 

^ Identical  for  each 
2/  Lowest  possible  value  of  r^)^. 
^ Hic^est  possible  value  of 
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EFFECTS  OF  INTERCORRELATION 


4 - Variable  Case:  ri2  = fia  = ri4  = .7;  r23=  r24  = .l;  N = 20 


-.2  0 .2  .4  .6  .8  1.0 


r34 

U.  S.  DEPARTMENT  OF  AGRICULTURE  NEG.  488-53  (11)  AGRICULTURAL  MARKETING  SERVICE 


Figure  8 


Table  11.-  Data  for  case  in  'vrtiich  ri?  = r^j  = ^lU  = 0"7> 
rg3  = rglj.  = 0.1,  and  N = 20 


r34 

^12. 3k 

P13.23 

P 14.23 

Sp 

y 

t -ratio  for 

P12.34 

P13.24 

P 14.23 

0.5765  2/ 

0.6191 

O.hOhS 

0.4048 

0.3079 

2.0105 

1.3145 

1.3l'+5 

.6000 

.6202 

.3987 

.3987 

.3133 

1.9799 

1.2728 

1.2728 

.7000 

.6250 

.3750 

.3750 

.3455 

1.0891 

1.0854 

1.0854 

.8000 

.6292 

.3539 

• 3539 

.4054 

1.5522 

.8731 

.8731 

.9000 

.6330 

.3351 

• 3351 

.5507 

1.1494 

.6085 

.6085 

.9500 

.63i)7 

.3261) 

.3264 

.7641 

.8307 

.4272 

.4272 

1.0000  2/ 

... 

— 

— 

... 

... 

... 

... 

l/  Identical  for  each  p . 

2/  Lowest  possible  value  of 
^ Highest  possible  value  of  r3i|.. 
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EFFECTS  OF  SNTERCORRELATION 


4 -Variable  Case:  ri2  = ri3  = ^14  = .5 ; r23=r24  = -5;  N = 20 


34  ) \ 
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\ 
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3.24=  ^ 

14.23 
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f^l2.34. 
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-.2  0 .2  .4  .6  .8  KO 
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Figure  9 


Table  12.-  Data 

for  case  in 

which 

/ 

= = 0.5, 

^23  = ^24 

=0.5,  and  N 

» 20 

%2.3h 

i ^3.24 

: ^14.23  1 

H ; 

t -ratio  for 

- 

y ; 

; 

^12. 34 

: ^13.24 
s 

: ® 14.23 

-0.3333  2/ 

-1.0000 

1.4999 

1.4999 

0.5303 

-1.8856 

2.8285 

2.8285 

- .3000 

- .7500 

1.2500 

1.2500 

.4586 

-1.6353 

2.7255 

2.7255 

- .2500 

- .5000 

1.0000 

1.0000 

.3873 

-1 .2910 

2.5820 

2.5820 

- .2000 

- .3333 

.8333 

.8333 

.3402 

- .9798 

2.4495 

2.4495 

- .1000 

- .1250 

.6250 

.6250 

.2827 

- .4422 

2.2111 

2.2111 

0 

0 

.5000 

.5000 

.2500 

0 

2.0000 

2.0000 

.1000 

.0833 

.4167 

.4167 

.2303 

.3618 

1.8091 

1.8091 

.2000 

.lJt29 

.3571 

.3571 

.2187 

.6532 

1.6329 

1.6329 

.3000 

.1875 

.3125 

.3125 

.2129 

.8806 

1.4676 

1.4676 

.IfOOO 

.2222 

.2778 

.2778 

.2122 

1.0474 

1.3093 

1.3093 

.5000 

.2500 

.2500 

.2500 

.2165 

1.1547 

1.1547 

1.1547 

.6000 

.2727 

.2273 

.2273 

.2273 

1.2000 

1.0000 

1,0000 

.7000 

.2917 

.2083 

.2083 

.2480 

1.1762 

.8401 

.8401 

.8000 

.3077 

.1923 

.1923 

.2885 

I.O667 

.6667 

.6667 

.9000 

.321^ 

.1786 

.1786 

.3892 

.8259 

.4588 

.4588 

1.0000  2/ 

... 

... 

l/  Identical  for  each  p. 

2/  Lowest  possible  value  of 
3/  Highest  possible  value  of 
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Figure  10 


Table  I3.-  Data  for  case  In  which  ri2  “ ^13  “ *'l4  ” ®*5> 
^23  ^24  ” N = 20 


P12.34 

P13.24 

® 14.23 

: ^ 

t-ratlo  for  - 

= ^12.34 

: ®13.24  : 

^14.23 

-0.3333  2/ 

0.5000 

0.7500 

0.7500 

0.2652 

1.8856 

2.8284 

2.8284 

- .3000 

.5000 

.7li*3 

.71*»3 

.2574 

1.9429 

2.7756 

2.7756 

- .2500 

.5000 

.6667 

.6667 

.2472 

2.0226 

2.69^ 

2.6968 

- .2000 

.5000 

.6250 

,6250 

.2387 

2.0949 

2.6186 

2.6186 

- .1000 

.5000 

.5556 

.5556 

.2255 

2.2172 

2.4636 

2.4636 

0 

.5000 

.5000 

.5000 

.2165 

2.3094 

2.3094 

2.3094 

.1000 

.5000 

.4546 

.4546 

.2109 

2.3708 

2.1553 

2.1553 

.2000 

.5000 

.4167 

.4167 

.2083 

2.4000 

2.0000 

2.0000 

.3000 

.5000 

.3846 

.3846 

.2088 

2.3950 

1.8423 

1.8423 

.4000 

.5000 

.3571 

.3571 

.2125 

2.3525 

1.6803 

1.6803 

.5000 

.5000 

.3333 

.3333 

.2205 

2.2678 

1.5118 

1.5118 

.6000 

.5000 

.3125 

.3125 

.2344 

2.1333 

1.3333 

1.3333 

.7000 

.5000 

.2941 

.2941 

.2582 

1.9363 

1.1390 

1.1390 

.8000 

.5000 

.2778 

.2778 

.3027 

1.6518 

.9177 

.9177 

.9000 

.5000 

.2632 

.2632 

.4109 

1.2170 

.6405 

.6405 

1.0000  iJ 

... 

... 

... 

... 

^ Identical  for  eachp. 

^ Lowest  possible  value  of  r^j^. 
2/  Highest  possible  value  of  r^j^ 
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APPENDIX 


Note  1 


Limits  Imposed  on  Values  of  tij  Given  Values  of  and  Tg- 


By  definition,  no  simple  or  multiple  correlation  coefficient  can  exceed 
1 in  absolute  value. 


Let  us  repeat  text  equation  (2),  as  follows, 
2 


1.23 


^12  ^13^  “ ^^12  ^13  ^23 


1 - r 


23 


(2) 


noting  that  must  lie  between  1 and  0, 


If  El  2^  * 1,  we  have 


, 2 2 2 ^ 

* ^23  “ ^12  ^ ^13  ^12  ^13  ^23* 


(2.1) 


Eearranglng  terms,  we  have 


rgj®  - 2 ^ ^ 


(2.2) 


and,  using  a standard  formula  of  elementary  algebra,  we  obtain 


^23  = ^12  ^13  * y (l"ri2^)  (l-ri3^).  (2.3) 

If  r^g  ■ 0,9  and  r^^  » 0.7,  as  In  figure  3>  ve  have 

1*23  = 0.63  + >/  (0.19)  (0.51)  = 0.63  i 0.3113; 

hence,  r_^  = O.3187  or  0,9^13.  Substituting  these  values  back  in  equation 
(2),  we  obtain  Ef  23  “ ^ each  case. 

Only  if  r,^  = r can  r reach  the  maximum  value  of  1.  But  when 
2 0 ^3  23 

^23  “ ®1.23  ~ ^12 » which  is,  in  general,  less  than  1.  This  is  shown  in 

figures  1 and  2. 


2k  - 


Thus,  if  = r = 0,7,  equation  (2,3)  gives  us 

13 

^23  ~ 0.1«-9  + 0.51  = 1 or  - 0.02. 
Substituting  “ 0.02  for  in  equation  (2)  we  obtain 

2^  ' 0-98  - 0.98  (-  O.Qg)  = 0.9996  • 1 . 
^ ^ 1 -(-o;02)2  0.9996 


But  if  we  substitute  = 1 we  obtain 


1.23 


0.98  - 0 98(1) 

1 -d") 


0 . 


an  Indeterminate  value. 

This  indeterminacy  can  be  resolved  by  applying  L*Hopltal's  rule,  2/ 
from  which  we  obtain 


Note  2.  Minimum  Values  of  Specified  Correlation  and  EegreBslon  Measures 


Assume  that  we  are  given  the  values  of  r^  and  r,»  and  wish  to  obtain  the 
values  of  rg-  at  which  various  coefficients  reach  their  minimum  values  within 
the  permissible  ranges  in  which  (1)  is  equal  to  or  less  than  1 and  (2) 

r2^  is  equal  to  or  less  than  1 in  absolute  value.  A necessary  condition  for 

a minimum  is  that  the  partial  derivative  with  respect  to  r of  a measure  that 
is  a function  of  rg^  -b©  zero. 

1,  Starting  with  equation  (2), 


1.23 


12 


+ r 


il 


2_ 


2r 


12  ^23 


1 - 


we  find  that  E 


2 

1.23 


reaches  a minimum  when 


(2) 


2/  See  pp,  15“16  of  Woods, 'Frederick  S,,  Advanced  Calculus,  new  ed., 
397  pp.»  lllus..  New  York,  193^ > or  other  standard  calculus  texts. 
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w©  30©  that  the  sign  Is  determined  hy  the  numerator  only. 


If  >-23  = ^ , ^,3,3  . 0; 


12 


if  r 


23  < f o > 0;  and 

ri2  13 


if  r, 


23  > -ii  , 


■ ^3.2 


< 0. 


3.  The  minima  for  the  partial  correlation  coefficients,  ^ ^3^2  2‘ 

respectively  are  given  at  the  points  where 


^23  = ^ 

(5.1) 

^12 

and 

r = ^12 

23  • 

^13 

(6.1) 

In  tables  3 through  6,  r^^  < r^  and  r^^  o minimum  in  the  permissible 

range  of  values  for  However,  r^g  ^ has  a minimum  at  the  level  indicated 

by  equation  (5.1) • 

When  r,^  = r--,  r « end  r ^ are  equal  and  reach  their  lowest  point  in 
12  13  12.3  13.2 

the  permissible  range  when  rg^  = 1, 

4.  Finally,  the  mlnimimi  value  of  the  standard  errors  of  the  beta 
coefficients  can  be  obtained  easily  from  formula  (7)> 


? = 

12.3  ^13.2 


M. 


(7) 


(1-^23^)^ 


When  E-  = 1,  So  =0  (provided  r ^ is  loss  than  1).  Thus,  in 
1.23  ^ 23 


.tables  3 through  6,  S 


Q = 0 at  two  points,  on©  at  each  end  of  the  permissible 
12.3 


range  of  values  of  r^.,.  in  figures  1 and  2,  Sa 

P12.3 


0 at  one  point,  the 


- 25  “ 


23 


(^12  ^13  ) ± (^12  - ri3^). 

' 2:,  o 

12  13 


(2.M 


that  Is,  when  = ^12 


or  ^13 


13 


^12 


o 

When  figures  1 and  2),  ^^^23  2°©aches  Its  minimum  value 

when  T22  = !•  When  r^^  ^ ^±2^  only  one  of  the  two  values  of  r2^  given  by 

equation  (2.U)  will  be  less  than  1,  and  hence  a permissible  value.  In  figure  3; 
we  have  r^o  = 0.7  ® 0,78;  in  the  data  shown  in  table  = 0,56;  aai  so  on. 

^ 0.9 

It  is  clear  that  these  are  minimum  rather  than  majclmum  values. 

2.  Using  the  same  approach,  the  minima  for  the  beta  coefficients, 

^12.3  ®“^^13,2,  r®sp©ctlvely,  are  given  at  the  points  where 


and 


■23 


■23 


^12  1 \/^12^  ~ ^13'^ 


13 


^13  -\/^13 


2 


12 


(3.2) 


i^.2) 


12 


If  r 


12 


r^^,  the  two  beta  coefficients  are  identical  and  reach  their 


low  point  when  rg^  = 1,  If  r^^^  > r^g^  the  low  point  forjS^^g  3 imaginary, 
while  that  for  ^^3  2 ^ value  somewhat  greater  than  ^13  . The 

^12 

converse  is  true  if  the  data  shown  in  table  for  example, 

^13.2  minimum  value  in  the  permissible  range;  ^22  Z ^ minimum  value 

at  the  point 

rpq  = 0.9  - sT  0*^1  “ 0.25  = 0.900  i 0.7^  = 0«304  . 

0.5  0.5 

The  second  value  of  rg^  is  outside  the  pemlssible  range, 

A prominent  feature  of  cases  3 to  6 is  the  fact  that^S^^  2 
at  a value  of  rg^  somewhat  greater  than  that  of  r^^*  Sef erring  to  equation  (4), 

^11.2  =■  liilliiikL. . 


1 - rg^a 


(•*) 
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lowest  pemlsslble  value  of  r. 

Figures  1 and  2 indicate  that,  if  approaches  infinity 

12  * 3 

as  rp-  approaches  1.  This  follows  readily  from  equation  (7),  since  if  "Pr  ig 

0 1 *23 

less  than  1 at  the  point  where  rp^  = 1,  we  have  a real  number  divided  by  zero. 

If  we  set  ^^12,3  = 0 we  obtain 

"■23 

*■23^  - ^^12  ''ll  I-,,®  + ZF(ri2®  + - 17  1-23  - r^2  r,,  = 0 . {7.3) 


For  given  values  of  r^^  and  r^^  this  can  be  solved  most  readily  by  plotting  the 
values  of  the  function  for  a range  of  values  of  r^^.  Thus,  for  table  5,  the 
expression  (7 *3)  becomes 

3 2 

^23  - 0.81  r23  + 0.8  r23  - 0.27  « 0 . 

This  equals  zero  when  r^^  is  approximately  0.1|.2U.  It  is  clear  from  the  table 
that  this  is  a maximum,  or  upper  turning  point,  rather  than  a minimum. 

Note  3»  Relation  of  Correlation  Formulas  in  Determinant  Notations  to  the 

Table  or  Inverse  Correlation  Matrix 

The  elements  of  the  inverse  of  a matrix  of  simple  correlation  coefficients 
may,  in  the  three -variable  case,  be  written  as  follows; 


(3) 


zi\ii 

-A  12 

Ai3 

A 

A 

-Ai2 

A 22 

-As^ 

A 

A 

A 

^13 

-^23 

A33 

The  array  of  elements  in  the  inverse  is  often  referred  to  as  " the  table" 
in  confutation  methods  such  as  those  developed  by  Waugh  U/.  In  this  notation, 
^11  ~ ^^^11  ^ ^12  “ ~ z^l2  , and  so  on, 

^ A 


Waugh,  F.V.  A samplified  Method  of  Determining  Multiple  Regression 
Constants,  Amer.  Statis,  Assoc.  Jour.  3^569^-700.  1935 • 
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To  calculate  ^ Inverse  or  table,  we  divide  -P^g  by  P^^. 

This  is  equivalent  to  equation  (3*); 


12.3 


"^12  = ^ 


12  . 


11 


4] 


42  . 
4i 


(3*) 


All  the  other  formulas  in  the  text  can  be  derived  in  the  same  fashion 


from  the  P table . For  example . 
^ J 


E 


1.23 


= 1-1  = 1- 


A 


-P 


12.3 


12 


-n 

= A 


12 


\/4Li'^g  'J\x  ’ Aa 


(2*) 


(5') 


and  so  on. 


The  text  formulas  in  determinant  notation  can  be  generalized  for  any 
number  of  variables.  The  same  is  true  of  the  corresponding  formulas  in  the 
notation.  Thus,  ^ 


12. 3^ 


A\i2  . 


(10) 


where  P^^  and  P^g  are  the 
U-rowed  determinant. 


first  and  second  elements  in  the  first  row  of 


a 


